1. Field of the Invention
The present invention relates to an image processing method for controlling a mobile robot, and more particularly, to a method for removing shadows from a captured image using two indoor surveillance cameras.
2. Description of the Related Art
In general, Video Surveillance (VS) is a computer vision technology that automatically processes video images of a camera installed to view specific areas of interest, for security or traffic management and such given purposes, and recently, it has been attracting the attention of many researchers.
The main use of such VS is to detect dynamic objects such as vehicles or people in video images, to classify the types of objects, and to analyze and understand motion.
The first step in VS is to extract the foreground in the image, and in the extracted foreground, other than the dynamic object which is the object of interest, its shadow is also included. At this time, the foreground appears different from the original form of the object, which can lead to errors in subsequent procedures such as classification or recognition. For this reason, a variety of techniques have been studied for detecting shadows in an image to eliminate the shadow (Non-Patent Document 1).
Prati et al. (Non-Patent Document 2) discloses that it has been reported that when existing shadow detection techniques were classified into a statistical parametric (SP) technique (Non-Patent Document 3) and a statistical non-parametric (SNP) technique (Non-Patent Document 4), and a deterministic model-based (DM) method (Non-Patent Document 5) and a deterministic non-model based (DNM) method, and they were experimented, the DNM method using color information showed the most stable and good results.
The DNM technique proposed by Cucchiara et al. (Non-Patent Document 6) uses a threshold value to detect a pixel value change by a shadow in an HSV (Hue Saturation Value) color space. The HSV color space is most similar to human color recognition, and the brightness V and the colors H and S are separated, which is advantageous for detecting shadow pixels.
Prati et al. compared conventional techniques for detecting shadows in video images. As a result, the DNM technique showed the most stable result among various techniques in various cases. The technique of Cucchiara et al. was a pixel-based DNM technique, and first, a RGB (Red Green Blue) color image of a camera was converted into an HSV color image according to the following Equations 21, 22, and 23.
                    H        =                  {                                                                                                                θ                      ,                                                                                                                          if                        ⁢                                                                                                  ⁢                        B                                            ≤                      G                                                                                                                                                          360                        -                        θ                                            ,                                                                                                                          if                        ⁢                                                                                                  ⁢                        B                                            >                      G                                                                                  ⁢                                                          ⁢              when              ⁢                                                          ⁢              θ                        =                                          cos                                  -                  1                                            ⁢                                                                    1                    2                                    ⁡                                      [                                                                  (                                                  R                          -                          G                                                )                                            +                                              (                                                  R                          -                          B                                                )                                                              ]                                                                                        [                                                                                            (                                                      R                            -                            G                                                    )                                                2                                            +                                                                        (                                                      R                            -                            B                                                    )                                                ⁢                                                  (                                                      G                            -                            B                                                    )                                                                                      ]                                                        1                    2                                                                                                          [                  Equation          ⁢                                          ⁢          21                ]                                S        =                  1          -                                    3                              (                                  R                  +                  G                  +                  B                                )                                      ⁡                          [                              min                ⁡                                  (                                      R                    ,                    G                    ,                    B                                    )                                            ]                                                          [                  Equation          ⁢                                          ⁢          22                ]                                V        =                              1            3                    ⁢                      (                          R              +              G              +              B                        )                                              [                  Equation          ⁢                                          ⁢          23                ]            
Then, in the number t-th frame of the converted video image, if any pixel p in the foreground satisfies all three conditions of Equation 24 below, it was determined to belong to the shadow region SH.
                                                                        p                ∈                                  sh                  ⁢                                                                          ⁢                  if                  ⁢                                                                          ⁢                                      C                    1                                                              ⩓                              C                2                            ⩓                              C                3                                      ⁢                                                  ⁢                                          when                ⁢                                                                  ⁢                                  C                  1                                ⁢                                  :                                ⁢                                                                  ⁢                α                            ≤                                                                    I                    V                    t                                    ⁡                                      (                    p                    )                                                                                        B                    V                    t                                    ⁡                                      (                    p                    )                                                              ≤              3                        ⁢                                                  ⁢                                          C                2                            ⁢                              :                            ⁢                                                          ⁢                                                I                  S                  t                                ⁡                                  (                  p                  )                                                      -                                          B                S                t                            ⁡                              (                p                )                                              ≤                      τ            S                          ⁢                                  ⁢                                            C              3                        ⁢                          :                                ⁢                                          |                                                    I                H                t                            ⁡                              (                p                )                                      -                                          B                H                t                            ⁡                              (                p                )                                              |                      ≤                          τ              H                                                          [                  Equation          ⁢                                          ⁢          24                ]            
Here, ^ means logical product, I(p) is a value at the foreground of a pixel p, and B(p) is a value at the background of a pixel p. Also, lower subscripts were used to denote the H, S, or V color values of a pixel, and α, β, τS and τH are the thresholds used for determination. These thresholds are appropriately determined by trial and error by a user so that the shadow pixels can be appropriately extracted from images used for learning. Equation 22 means that although the V value of the pixel changes somewhat greatly within a certain range, when the values of S and H do not change greatly, it is judged that that pixel is shadowed. In other words, shadows cause the brightness of the pixel to be significantly lowered without greatly changing the color of the pixel.
When it comes to location recognition and control of an image-based mobile robot, the recognition and removal of shadows occurring in images are very important. Shadows created by objects in an environment where the mobile robot moves cause incorrect feature point extractions or image analysis errors. Generally, for shadow removal of objects, a background image removal method is widely used. This method has high accuracy but is disadvantageous in that it requires a large amount of calculation. In addition, since the shadow component produced by indoor illumination remains on the image with the background removed, it needs to be further removed to obtain accurate object information. Therefore, many studies for shadow removal are focused on developing algorithms with high computation speed.
So far, mainly two researches for removing shadows from a background area, using spatial information and using color information, are being widely used. The method of using spatial information judges the shadow by using the property that the shadow has a flatter boundary than the moving object. This method has a disadvantage that although it is possible to remove the shadow with a relatively high accuracy, it is time-consuming because it requires a large amount of calculation. Since a real-time video surveillance system needs to handle advanced functions such as tracing moving objects for several tens of frames per second or more, in the end, a large amount of computation acts as a great burden. As a method capable of solving the problem of the large amount of calculation, there is a method of using color information. This method is based on the fact that the shadow region has the same color as the background model but a difference in brightness occurs. The method using color information is suitable for a real-time video surveillance system because it can remove shadows with only a relatively simple operation.
As prior arts, there are Non-Patent Document 1: M. Hwang and D. J. Kang, “A shadow region suppression method using intensity projection and converting energy to improve the performance of probabilistic background subtraction,” Journal of Institute of Control, Robotics and Systems, vol. 16, no. 1, pp. 69-76, 2010; Non-Patent Document 2: A. Prati and R. Cucchiara, & quot; Analysis and detection of shadows in video streams: a comparative evaluation, & quot; Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 571-576, 2001, Non-Patent Document 3: I. Mikic, P. C. Cosman, G. T. Kogut, and M. M. Trivedi, “Moving shadow and object detection in traffic scenes,” Proc. International Conference on Pattern Recognition, vol. 1, pp. 321-324, 2000; Non-Patent Document 4: T. Horprasert, D. Harwood, and L. Davis, “A statistical approach for real-time robust background subtraction and shadow detection,” Proc. IEEE International Conference on Computer Vision, pp. 1-19, 1999; Non-Patent Document 5: M. Kilger, & quot; A shadow handler in a video-based real-time traffic monitoring system, & quot; Proc. IEEE Workshop on Applications of Computer Vision, pp. 11-18, 1992; Non-Patent Document 6: Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting objects, shadows and ghosts in video streams by exploiting color and motion information,” Proc. IEEE International Conference on Image Analysis and Processing, pp. 360-365, 2001; and Non-Patent Document 7: T. Collins, A J Lipton, T. Kanade, H. Fujiyoshi, D. Duggins, Y. Tsin, D. Tolliver, N. Enomoto, O. Hasegawa, P. Burt, and L. Wixson, A system for video surveillance and Monitoring, Carnegie Mellon University Robotics Institute Technical Report CMU-RI-TR-00-12, 2000.